Feedforward Neural Network Architecture Research Articles

Artificial Intelligence, Lymphoid Neoplasms, and Prediction of MYC, BCL2, and BCL6 Gene Expression Using a Pan-Cancer Panel in Diffuse Large B-Cell Lymphoma

Background: Artificial intelligence in medicine is a field that is rapidly evolving. Machine learning and deep learning are used to improve disease identification and diagnosis, personalize disease treatment, analyze medical images, evaluate clinical trials, and speed drug development. Methods: First, relevant aspects of AI are revised in a comprehensive manner, including the classification of hematopoietic neoplasms, types of AI, applications in medicine and hematological neoplasia, generative pre-trained transformers (GPTs), and the architecture and interpretation of feedforward neural net-works (multilayer perceptron). Second, a series of 233 diffuse large B-cell lymphoma (DLBCL) patients treated with rituximab-CHOP from the Lymphoma/Leukemia Molecular Profiling Project (LLMPP) was analyzed. Results: Using conventional statistics, the high expression of MYC and BCL2 was associated with poor survival, but high BCL6 was associated with a favorable overall survival of the patients. Then, a neural network predicted MYC, BCL2, and BCL6 with high accuracy using a pan-cancer panel of 758 genes of immuno-oncology and translational research that includes clinically relevant actionable genes and pathways. A comparable analysis was performed using gene set enrichment analysis (GSEA). Conclusions: The mathematical way in which neural networks reach conclusions has been considered a black box, but a careful understanding and evaluation of the architectural design allows us to interpret the results logically. In diffuse large B-cell lymphoma, neural networks are a plausible data analysis approach.

Enhancing Spam Detection: A Crow-Optimized FFNN with LSTM for Email Security

Email security is paramount in today's digital landscape, as the proliferation of spam emails poses a significant threat to individuals and organizations alike. To combat this menace, this study introduces a novel approach that marries the power of Crow Search Optimization (CSO) with a Feedforward Neural Network (FFNN) and Long Short-Term Memory (LSTM) architecture to bolster spam detection. The proposed Crow-Optimized FFNN with LSTM (C-FFNN-LSTM) leverages CSO to fine-tune the neural network's parameters, optimizing its ability to distinguish between legitimate emails and spam. The CSO algorithm mimics the collaborative behavior of crows, thereby enhancing the model's convergence and robustness. Experimental results showcase the effectiveness of the C-FFNN-LSTM approach, achieving remarkable accuracy rates and reducing false positives. This innovation not only enhances email security but also offers a promising avenue for refining spam detection algorithms across various domains. In an era of ever-evolving cyber threats, the C-FFNN-LSTM framework stands as a beacon of improved email security, safeguarding digital communication channels.
 In our methodology, we attained an outstanding accuracy level of 99.1% during the testing phase.

Creating a computer assisted ICD coding system: Performance metric choice and use of the ICD hierarchy

Objective:Machine learning methods hold the promise of leveraging available data and generating higher-quality data while alleviating the data collection burden on healthcare professionals. International Classification of Diseases (ICD) diagnoses data, collected globally for billing and epidemiological purposes, represents a valuable source of structured information. However, ICD coding is a challenging task. While numerous previous studies reported promising results in automatic ICD classification, they often describe input data specific model architectures, that are heterogeneously evaluated with different performance metrics and ICD code subsets.This study aims to explore the evaluation and construction of more effective Computer Assisted Coding (CAC) systems using generic approaches, focusing on the use of ICD hierarchy, medication data and a feed forward neural network architecture. Methods:We conduct comprehensive experiments using the MIMIC-III clinical database, mapped to the OMOP data model. Our evaluations encompass various performance metrics, alongside investigations into multitask, hierarchical, and imbalanced learning for neural networks. Results:We introduce a novel metric, ▪ , tailored to the ICD coding task, which offers interpretable insights for healthcare informatics practitioners, aiding them in assessing the quality of assisted coding systems. Our findings highlight that selectively cherry-picking ICD codes diminish retrieval performance without performance improvement over the selected subset. We show that optimizing for metrics such as NDCG and AUPRC outperforms traditional F1-based metrics in ranking performance. We observe that Neural Network training on different ICD levels simultaneously offers minor benefits for ranking and significant runtime gains. However, our models do not derive benefits from hierarchical or class imbalance correction techniques for ICD code retrieval. Conclusion:This study offers valuable insights for researchers and healthcare practitioners interested in developing and evaluating CAC systems. Using a straightforward sequential neural network model, we confirm that medical prescriptions are a rich data source for CAC systems, providing competitive retrieval capabilities for a fraction of the computational load compared to text-based models. Our study underscores the importance of metric selection and challenges existing practices related to ICD code sub-setting for model training and evaluation.

The Role of cfDNA Biomarkers and Patient Data in the Early Prediction of Preeclampsia: Artificial Intelligence Model

ObjectiveAccurate individualized assessment of preeclampsia risk enables the identification of patients most likely to benefit from initiation of low-dose aspirin at 12-16 weeks’ gestation when there is evidence for its effectiveness, as well as guiding appropriate pregnancy care pathways and surveillance. The primary objective of this study was to evaluate the performance of artificial neural network models for the prediction of preterm preeclampsia (<37 weeks’ gestation) using patient characteristics available at the first antenatal visit and data from prenatal cell-free DNA (cfDNA) screening. Secondary outcomes were prediction of early onset preeclampsia (<34 weeks’ gestation) and term preeclampsia (≥37 weeks’ gestation). MethodsThis secondary analysis of a prospective, multicenter, observational prenatal cfDNA screening study (SMART) included singleton pregnancies with known pregnancy outcomes. Thirteen patient characteristics that are routinely collected at the first prenatal visit and two characteristics of cfDNA, total cfDNA and fetal fraction (FF), were used to develop predictive models for early-onset (<34 weeks), preterm (<37 weeks), and term (≥37 weeks) preeclampsia. For the models, the ‘reference’ classifier was a shallow logistic regression (LR) model. We also explored several feedforward (non-linear) neural network (NN) architectures with one or more hidden layers and compared their performance with the LR model. We selected a simple NN model built with one hidden layer and made up of 15 units. ResultsOf 17,520 participants included in the final analysis, 72 (0.4%) developed early onset, 251 (1.4%) preterm, and 420 (2.4%) term preeclampsia. Median gestational age at cfDNA measurement was 12.6 weeks and 2,155 (12.3%) had their cfDNA measurement at 16 weeks’ gestation or greater. Preeclampsia was associated with higher total cfDNA (median 362.3 versus 339.0 copies/ml cfDNA; p<0.001) and lower FF (median 7.5% versus 9.4%; p<0.001). The expected, cross-validated area under the curve (AUC) scores for early onset, preterm, and term preeclampsia were 0.782, 0.801, and 0.712, respectively for the LR model, and 0.797, 0.800, and 0.713, respectively for the NN model. At a screen-positive rate of 15%, sensitivity for preterm preeclampsia was 58.4% (95% CI 0.569, 0.599) for the LR model and 59.3% (95% CI 0.578, 0.608) for the NN model.The contribution of both total cfDNA and FF to the prediction of term and preterm preeclampsia was negligible. For early-onset preeclampsia, removal of the total cfDNA and FF features from the NN model was associated with a 6.9% decrease in sensitivity at a 15% screen positive rate, from 54.9% (95% CI 52.9-56.9) to 48.0% (95% CI 45.0-51.0). ConclusionRoutinely available patient characteristics and cfDNA markers can be used to predict preeclampsia with performance comparable to other patient characteristic models for the prediction of preterm preeclampsia. Both LR and NN models showed similar performance.

Algorithms of the Möbius function by random forests and neural networks

The Möbius function μ(n)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu (n)$$\\end{document} is known for containing limited information on the prime factorization of n. Its known algorithms, however, are all based on factorization and hence are exponentially slow on logn\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\log n$$\\end{document}. Consequently, a faster algorithm of μ(n)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu (n)$$\\end{document} could potentially lead to a fast algorithm of prime factorization which in turn would throw doubt upon the security of most public-key cryptosystems. This research introduces novel approaches to compute μ(n)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu (n)$$\\end{document} using random forests and neural networks, harnessing the additive properties of μ(n)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu (n)$$\\end{document}. The machine learning models are trained on a substantial dataset with 317,284 observations (80%), comprising five feature variables, including values of n within the range of 4×109\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$4\ imes 10^9$$\\end{document}. We implement the Random Forest with Random Inputs (RFRI) and Feedforward Neural Network (FNN) architectures. The RFRI model achieves a predictive accuracy of 0.9493, a recall of 0.5865, and a precision of 0.6626. On the other hand, the FNN model attains a predictive accuracy of 0.7871, a recall of 0.9477, and a precision of 0.2784. These results strongly support the effectiveness and validity of the proposed algorithms.

Image‐based characterization of flocculation processes through PLS inspired representation learning in convolutional neural networks

AbstractMonitoring of flocculation processes such as those used in downstream processing of a fermentation broth is essential for process control. One approach is to apply microscopic imaging combined with image analysis for characterizing the state of the process. In this work, we investigate and compare the use of supervised feedforward convolutional neural network (CNN) architectures to predict the process states from the image information and compare the results with the traditional alternative of characterizing flocs based on manually engineered image features guided by human expertise. From a well‐defined image data set representing six process states, the objective is to establish end‐to‐end classification models which are accurate but at the same time learn meaningful latent variable space representations. Specifically, we evaluate three different CNN architectures with varying degrees of regularization and compare results with logistic regression models based on inputs from two different traditional feature engineering methods. By applying global average pooling as a structural regularizer to the CNN architecture, we significantly improve the generalization performance in comparison with the classification accuracies of the traditional feature engineered models. Furthermore, we show that by imposing a projection to latent structures (PLS) like regularization framework onto the CNN, it can also learn a latent variable representation that mimics the features selected by human expertise.

A Bidirectional Feedforward Neural Network Architecture Using the Discretized Neural Memory Ordinary Differential Equation.

Deep Feedforward Neural Networks (FNNs) with skip connections have revolutionized various image recognition tasks. In this paper, we propose a novel architecture called bidirectional FNN (BiFNN), which utilizes skip connections to aggregate features between its forward and backward paths. The BiFNN accepts any FNN as a plugin that can incorporate any general FNN model into its forward path, introducing only a few additional parameters in the cross-path connections. The backward path is implemented as a nonparameter layer, utilizing a discretized form of the neural memory Ordinary Differential Equation (nmODE), which is named [Formula: see text]-net. We provide a proof of convergence for the [Formula: see text]-net and evaluate its initial value problem. Our proposed architecture is evaluated on diverse image recognition datasets, including Fashion-MNIST, SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet. The results demonstrate that BiFNNs offer significant improvements compared to embedded models such as ConvMixer, ResNet, ResNeXt, and Vision Transformer. Furthermore, BiFNNs can be fine-tuned to achieve comparable performance with embedded models on Tiny-ImageNet and ImageNet-1K datasets by loading the same pretrained parameters.

Deep Ensemble Transformers for Dimensionality Reduction.

We propose deep ensemble transformers (DETs), a fast, scalable approach for dimensionality reduction problems. This method leverages the power of deep neural networks and employs cascade ensemble techniques as its fundamental feature extraction tool. To handle high-dimensional data, our approach employs a flexible number of intermediate layers sequentially. These layers progressively transform the input data into decision tree predictions. To further enhance prediction performance, the output from the final intermediate layer is fed through a feed-forward neural network architecture for final prediction. We derive an upper bound of the disparity between the generalization error and the empirical error and demonstrate that it converges to zero. This highlights the generalizability of our method to parameter estimation and feature selection problems. In our experimental evaluations, DETs outperform existing models in terms of prediction accuracy, representation learning ability, and computational time. Specifically, the method achieves over 95% accuracy in gene expression data and can be trained on average 50% faster than traditional artificial neural networks (ANNs).

The Application of Feed - Forward Neural Network Architecture for Improving Energy Efficiency

The energy sector contributes approximately two-thirds of global greenhouse gas emissions. In this context, the sector must adapt to new supply and demand networks for all future energy sources. The ongoing transformation in the European energy field is driven by the ambition of the European Union to reach the climate objectives set for 2030. The main actions are increasing renewable energy production, adapting transition fuels like natural gas to reduce emissions, improving energy efficiency across all economic sectors, prioritizing building, transportation, and industry, developing Carbon Capture and Storage technologies, and ensuring universal access to clean and affordable energy. The significant changes envisaged in the energy sector to increase renewable energy production and consumption require improved integration and more use of predictive tools to support stakeholders' decision-making processes. This article presents a case study to assess the performance of predictive models based on a Feed-Forward Neural Network architecture that employ Root Mean Squared Propagation as their optimization function in terms of choosing the most appropriate activation function for this kind of data input and output. The training and testing phases of the models use data about building energy consumption. The lowest training time and Root Mean Square Error values were similar for Rectified Linear Unit and Tanh activation functions. The model with the Rectified linear Unit activation function performed best.

A systematic comparison of different machine learning models for the spatial estimation of air pollution

Air pollutants harm human health and the environment. Nowadays, deploying an air pollution monitoring network in many urban areas could provide real-time air quality assessment. However, these networks are usually sparsely distributed and the sensor calibration problems that may appear over time lead to missing and wrong measurements. There is an increasing interest in developing air quality modelling methods to minimize measurement errors, predict spatial and temporal air quality, and support more spatially-resolved health effect analysis. This research aims to evaluate the ability of three feed-forward neural network architectures for the spatial prediction of air pollutant concentrations using the measures of an air quality monitoring network. In addition to these architectures, Support Vector Machines and geostatistical methods (Inverse Distance Weighting and Ordinary Kriging) were also implemented to compare the performance of neural network models. The evaluation of the methods was performed using the historical values of seven air pollutants (Nitrogen monoxide, Nitrogen dioxide, Sulphur dioxide, Carbon monoxide, Ozone, and particulate matters with size less than or equal to 2.5 μ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\upmu $$\\end{document}m and to 10 μ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\upmu $$\\end{document}m) from an urban air quality monitoring network located at the metropolitan area of Madrid (Spain). To assess and compare the predictive ability of the models, three estimation accuracy indicators were calculated: the Root Mean Squared Error, the Mean Absolute Error, and the coefficient of determination. FFNN-based models are superior to geostatistical methods and slightly better than Support Vector Machines for fitting the spatial correlation of air pollutant measurements.Graphical abstract

Prediction of intensified ethanol fermentation of sugarcane using a deep learning soft sensor and process analytical technology

AbstractBACKGROUNDIntensified ethanol fermentation produces higher ethanol concentrations while reducing water and energy requirements. Nevertheless, the inhibitory and detrimental effect of the cellular stress barriers in this process further complicates the nonlinear dynamic relationship between the variables that directly reflect the fermentation quality. These key variables are hard to measure in real time and therefore cannot be directly controlled.RESULTSThis work presents the development of a soft sensor that predicts in real time the ethanol and substrate concentrations of an intensified fermentation. The soft sensor uses feedforward neural networks (FNNs) with easily measurable process analytical technology (PAT) tools. The application of advanced PAT tools such as redox potential and capacitance, in addition to temperature and pH are explored as input variables. The complex kinetic relationship between the studied variables was captured with FNN architectures with a single hidden layer and between 95 and 175 hidden neurons for the different cases studied. Acceptable predictions are achieved for the concentration of ethanol (RMSE = 9.5 and R2 = 0.97) and substrate (RMSE = 17.02 and R2 = 0.92).CONCLUSIONSThe results confirm that the proposed soft sensor can accurately predict the ethanol and substrate concentrations. Collectively, capacitance, redox potential, temperature and pH provide a powerful platform of PAT tools that can directly infer key variables showing the fermentation quality in real time. This study provides a significant step towards the systematic development of a reliable soft sensor with integration of advanced PAT tools. © 2023 Society of Chemical Industry.

Defining the optimal conditions using FFNNs and NARX neural networks for modelling the extraction of Sc from aqueous solution by Cryptand-2.2.1 and Cryptand-2.1.1

The main aim of this study is to figure out how well cryptand-2.2.1 (C 2.2.1) and cryptand-2.1.1 (C 2.1.1) macrocyclic compounds (MCs) work as novel extractants for scandium (Sc) by using an artificial neural network (ANN) models in MATLAB software. Moreover, C2.2.1 and C2.1.1 have never been evaluated to recover Sc. The independent variables impacting the extraction process (concentration of MC, concentration of Sc, pH, and time), and a nonlinear autoregressive network with exogenous input (NARX) and feed-forward neural network (FFNN) models were used to estimate their optimum values. The greatest obstacle in the selective recovery process of the REEs is the similarity in their physicochemical properties, specifically their ionic radius. The recovery of Sc from the aqueous solution was experimentally evaluated, then the non-linear relationship between those parameters was predictively modeled using (NARX) and (FFNN). To confirm the extraction and stripping efficiency, an atomic absorption spectrophotometer (AAS) was employed. The results of the extraction investigations show that, for the best conditions of 0.008 mol/L MC concentration, 10 min of contact time, pH 2 of the aqueous solution, and 75 mg/L Sc initial concentration, respectively, the C 2.1.1 and C 2.2.1 extractants may reach 99 % of Sc extraction efficiency. Sc was recovered from a multi-element solution of scandium (Sc), yttrium (Y), and lanthanum (La) under these circumstances. Whereas, at a concentration of 0.3 mol/L of hydrochloric acid, the extraction of Sc was 99 %, as opposed to Y 10 % and La 7 %. The Levenberg-Marquardt training algorithm had the best training performance with an mean-squared-error, MSE, of 5.232x10−6 and 6.1387x10−5 for C 2.2.1 and C 2.1.1 respectively. The optimized FFNN architecture of 4-10-1 was constructed for modeling recovery of Sc. The extraction process was well modeled by the FFNN with an R2 of 0.999 for the two MC, indicating that the observed Sc recovery efficiency consistent with the predicted one.

Modeling of diesel particulate filter temperature dynamics during exotherm using neural networks

Diesel Particulate Filter (DPF) in the diesel engine exhaust stream needs frequent regeneration (exotherm) to remove captured particulate matter (PM, or soot) without damaging to the porous DPF structure by controlling the peak temperatures and temperature gradients across the DPF. In this study, temperature distribution in a DPF is measured at 42 strategic locations in the test DPF under various regeneration conditions of exhaust flow rates, regeneration temperatures and soot loads. Then a data-based model with feed-forward neural network architecture is designed to model the thermal gradients and temperature dynamics of the DPF during the regeneration process. The neural network feature vector selection, network architecture, hyperparameter calibration process, measured data preprocessing, and experimental data acquisition procedure are evaluated. Over 7,400 experimental data points at various regeneration temperatures, flow rates and soot loads are used in training and validating the neural network model. It is found that the neural network model can accurately predict the 42 DPF bed temperatures simultaneously at different locations, and the time series analysis of both model-predicted and experimentally measured temperatures shows a good correlation. This indicates that the currently developed neural network model can provide spatial distribution of temperature in the DPF, and comprehend the nonlinearity of the temperature dynamics due to DPF soot load at exothermic conditions. These results demonstrate that the data-based model has capability in predicting thermal gradients within a DPF, aiding in determining a safer DPF regeneration strategy, onboard diagnostics and DPF development.

Information system of multi-stage analysis of the building of object models on a construction site

This study focuses on the development of a multi-stage analysis of building object models (BOM) on a construction site for modeling an “evolutionary” digital twin, by integrating building information modeling (BIM) technology and an artificial intelligence system. The concepts of photo modeling of the construction site using a group of moving cameras were outlined, as well as the possibility of integrating IoT technologies. The dynamic transition of real building structures into intermediate BIM representations of digital twins was investigated, with the prospect of enabling augmented reality technology. An artificial intelligence system combining Convolutional Neural Network (CNN) and Feed Forward Neural Network (FFNN) architectures has been developed as a comprehensive mechanism for the detection, categorization, and evaluation of BIM projects at all stages of their life cycle. The paper addresses the scaling prospects for the development of point cloud and mesh models, as well as the use of big data technology while optimizing the representation of the “evolutionary” BIM project of the digital twin of the construction site. The effectiveness of site conformance detection during the step-by-step construction of a BIM model, which shows consistency and provides a quantitative assessment of the processes occurring on the site, has been determined. The results of this research can be used to improve BIM modeling methods and concepts, in particular towards a multi-stage “evolutionary” representation of the digital twin of the construction site.

Neural network process simulations support a distributed memory system and aid design of a novel computer adaptive digital memory test for preclinical and prodromal Alzheimer's disease.

Growing evidence supports the importance of learning as a central deficit in preclinical/prodromal Alzheimer's disease. The aims of this study were to conduct a series of neural network simulations to develop a functional understanding of a distributed, nonmodular memory system that can learn efficiently without interference. This understanding is applied to the development of a novel digital memory test. Simulations using traditional feed forward neural network architectures to learn simple logic problems are presented. The simulations demonstrate three limitations: (a) inefficiency, (b) an inability to learn problems consistently, and (c) catastrophic interference when given multiple problems. A new mirrored cascaded architecture is introduced to address these limitations, with support provided by a series of simulations. The mirrored cascaded architecture demonstrates efficient and consistent learning relative to feed forward networks but also suffers from catastrophic interference. Addition of context values to add the capability of distinguishing features as part of learning eliminates the problem of interference in the mirrored cascaded, but not the feed forward, architectures. A mirrored cascaded architecture addresses the limitations of traditional feed forward neural networks, provides support for a distributed memory system, and emphasizes the importance of context to avoid interference. These process models contributed to the design of a digital computer-adaptive word list learning test that places maximum stress on the capability to distinguish specific episodes of learning. Process simulations provide a useful method of testing models of brain function and contribute to new approaches to neuropsychological assessment. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

AMITE: A Novel Polynomial Expansion for Analyzing Neural Network Nonlinearities.

Polynomial expansions are important in the analysis of neural network nonlinearities. They have been applied thereto addressing well-known difficulties in verification, explainability, and security. Existing approaches span classical Taylor and Chebyshev methods, asymptotics, and many numerical approaches. We find that, while these have useful properties individually, such as exact error formulas, adjustable domain, and robustness to undefined derivatives, there are no approaches that provide a consistent method, yielding an expansion with all these properties. To address this, we develop an analytically modified integral transform expansion (AMITE), a novel expansion via integral transforms modified using derived criteria for convergence. We show the general expansion and then demonstrate an application for two popular activation functions: hyperbolic tangent and rectified linear units. Compared with existing expansions (i.e., Chebyshev, Taylor, and numerical) employed to this end, AMITE is the first to provide six previously mutually exclusive desired expansion properties, such as exact formulas for the coefficients and exact expansion errors. We demonstrate the effectiveness of AMITE in two case studies. First, a multivariate polynomial form is efficiently extracted from a single hidden layer black-box multilayer perceptron (MLP) to facilitate equivalence testing from noisy stimulus-response pairs. Second, a variety of feedforward neural network (FFNN) architectures having between three and seven layers are range bounded using Taylor models improved by the AMITE polynomials and error formulas. AMITE presents a new dimension of expansion methods suitable for the analysis/approximation of nonlinearities in neural networks, opening new directions and opportunities for the theoretical analysis and systematic testing of neural networks.

Frontiers of thermobarometry: GAIA, a novel Deep Learning-based tool for volcano plumbing systems

The anatomy of the plumbing system of active volcanoes is fundamental to understand how magma is stored and channeled to the surface. Reliable geothermobarometric estimates are, therefore, critical to assess the depths and temperatures of the complex system of magmatic reservoirs that form a volcano apparatus. Here, we developed a novel Machine Learning approach (named GAIA, Geo Artificial Intelligence thermobArometry) based upon Feedforward Neural Networks to estimate P-T conditions of magma (clinopyroxene) storage and migration within the crust. Our Feedforward Neural Network method applied to clinopyroxene compositions yields better uncertainties (Root-Mean-Square Error and R2 score) than previous Machine Learning methods and set the basis for a novel generation of reliable geothermobarometers, which extends beyond the paradigm associated to crystal-liquid equilibrium. Also, the bootstrap procedure, inherent to the Feedforward Neural Network architecture, permits to perform a rigorous assessment of the P-T uncertainty associated to each clinopyroxene composition, as opposed to the Root-Mean-Square Error representing the P-T uncertainty of whole set of clinopyroxene compositions.As a test, we applied GAIA to assess P-T conditions of five Italian volcanoes (Somma-Vesuvius, Campi Flegrei, Etna, Stromboli, Volcano), which are among the most dangerous volcanic centres in Europe. The results on the depths of the plumbing systems are in excellent agreement with those obtained with independent geophysical and geodetic surveys, and provide further evidence to current models of volcano plumbing systems consisting of physically-separated reservoirs interconnected by a network of conduits channelling magma en route to the surface. The results on the magma (clinopyroxene crystallization) temperatures are also in agreement with other estimates, albeit obtained considering - mainly but not only - thermodynamically-based clinopyroxene-liquid geothermometers.GAIA can set robust estimates of magma storage, segregation, and ascent conditions within the plumbing system of active volcanoes, helping to unravel P-T variations, if any, during their eruptive history and providing robust clues to volcanic hazard assessment.

A comparative study on different neural network architectures to model inelasticity

AbstractThe mathematical formulation of constitutive models to describe the path‐dependent, that is, inelastic, behavior of materials is a challenging task and has been a focus in mechanics research for several decades. There have been increased efforts to facilitate or automate this task through data‐driven techniques, impelled in particular by the recent revival of neural networks (NNs) in computational mechanics. However, it seems questionable to simply not consider fundamental findings of constitutive modeling originating from the last decades research within NN‐based approaches. Herein, we propose a comparative study on different feedforward and recurrent neural network architectures to model 1D small strain inelasticity. Within this study, we divide the models into three basic classes: black box NNs, NNs enforcing physics in a weak form, and NNs enforcing physics in a strong form. Thereby, the first class of networks can learn constitutive relations from data while the underlying physics are completely ignored, whereas the latter two are constructed such that they can account for fundamental physics, where special attention is paid to the second law of thermodynamics in this work. Conventional linear and nonlinear viscoelastic as well as elastoplastic models are used for training data generation and, later on, as reference. After training with random walk time sequences containing information on stress, strain, and—for some models—internal variables, the NN‐based models are compared to the reference solution, whereby interpolation and extrapolation are considered. Besides the quality of the stress prediction, the related free energy and dissipation rate are analyzed to evaluate the models. Overall, the presented study enables a clear recording of the advantages and disadvantages of different NN architectures to model inelasticity and gives guidance on how to train and apply these models.

Information system based on multi-value classification of fully connected neural network for construction management

This study is devoted to solving the problem to determine the professional adaptive capabilities of construction management staff using artificial intelligence systems. It is proposed fully connected feed-forward neural network (FCF-FNN) architecture and performed empirical modeling to create a data set. Model of artificial intelligence system allows evaluating the processes in an FCF-FNN during the execution of multi-value classification of professional areas. A method has been developed for the training process of a machine learning model, which reflects the internal connections between the components of an artificial intelligence system that allow it to “learn” from training data. To train the neural network, a data set of 35 input parameters and 29 output parameters was used; the amount of data in the set is 936 data lines. Neural network training occurred in the proportion of 10% and 90%, respectively. Results of this study research can be used to further improve the knowledge and skills necessary for successful professional realization.

Multi-cancer classification; An analysis of neural network models

Background:Cancer identification is generally framed as binary classification, normally discrimination of a control group from a single cancer group. However, such models lack any cancer-specific information, as they are only trained on one cancer type. The models fail to account for competing cancer risks. Pan-cancer evaluation requires a model trained on multiple cancer types, and controls, simultaneously, so that a physician can be directed to the correct area of the body for further testing. Methods:We investigate neural network models to address multi-cancer classification problems across several data types commonly applied in cancer prediction, including circulating miRNA expression, protein, and mRNA. In particular, we present an analysis of neural network depth and type, and investigate how this relates to classification performance. In our comparisons, we include several state-of-the-art neural networks from the literature. We provide details on the optimal network depth and type, the activation functions and layer sizes. Results: Our analysis evidences that shallow (i.e., 1 or 2 layer), feed-forward neural network architectures offer greater performance in terms of mean sensitivity and precision when compared to deeper (i.e., >2 layer) feed-forward models, Convolutional Neural Network (CNN), and Graph CNN (GCNN) architectures, across a range of measurement technologies in cancer prediction (e.g., miRNA, mRNA and protein). We also discover that hyperbolic tangent activation functions offer the most consistent performance, and the optimal feed-forward models have descending layer size structure.